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Abstract 



This paper extends Information Theoretic analysis to packet random access systems with bursty traffic. A new 

oij; 

^ , channel coding approach for coding within each data packet is proposed with built-in support for bursty traffic 



phenomena, such as message underflow, and for random access mechanism, such as packet collision detection. 
The coding approach does not require joint communication rate determination either between the transmitters or 
between the transmitters and the receiver Its performance limitation is characterized by an achievable region defined 
in terms of communication rates, such that reliable packet recovery is supported for all rates within the region 
^ ■ and reliable collision detection is supported for all rates outside the region. For random access communication 

over a discrete-time memoryless channel using a class of random coding schemes, it is shown that the achievable 
I rate region of the introduced coding approach equals the Shannon information rate region without a convex hull 

CO , operation. Further connections between the achievable rate region and the Shannon information rate region are 

l/^ ■ explained. 
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I. Introduction 

Classical information theory regards each transmitter in a multiuser communication system as back- 
logged with an infinite reservoir of traffic [1][2]. To achieve reliable communication, transmitters first 
jointly determine their codebooks and information rates and share this information with the receiver and 
with each other. The encoded symbols are then transmitted to the receiver continuously over a long time 
duration. Channel capacity and channel coding theorems are proved using the standard argument of jointly 
typical sequences by taking the sequence (or codeword) length to infinity [1][2]. 
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By allowing a small acceptable communication error probability, information theoretic results can be ex- 
tended to channel coding within a long, but finite-length, time duration [3]. Consequently, in a time-slotted 
communication model, if information bits arrive stochastically and queue at the transmitters, transmitters 
can jointly adapt their information rates in each time slot to optimize certain system performance based on 
coding theoretic results and on the status of the message queues [4] [5] [6]. Determination of fundamental 
performance limitations, such as the throughput and queuing delay tradeoff, can therefore be obtained as 
in [5]. Although such an extension enabled the incorporation of stochastic traffic arrival in information 
theoretic analysis, it inherited the key assumption, and hence also the limitation, of joint information rate 
determination among transmitters in each time slot [4] [5] [6]. 

For various practical reasons such as bursty traffic arrival, timely data dissemination, and cognitive 
networking [7], transmitters in a communication network may not always want to or be able to jointly 
design chaimel codes and determine their communication rates. Random chaimel access is therefore 
commonly seen in practical networks [8]. In random access communication, transmitters make independent 
channel access decisions, such as whether or not to transmit a packet, without sharing these decisions 
among each other or with the receiver. If we regard the idling of a transmitter as setting its communication 
rate at zero and regard the transmission of a packet as setting the communication rate of a transmitter at 
a non-zero value, then communication rates of the transmitters are determined independently, and the rate 
information is not shared with the receiver. Independent rate determination leads to unavoidable packet 
collision [9]. When the joint rates of the transmitters are chosen such that reliable packet recovery is not 
possible, for efficient functioning of the upper layer protocols, the receiver is required to report a packet 
collision rather than blindly forwarding unreliable messages to the upper layers [9]. 

Due to the challenges of relaxing the joint rate determination assumption among transmitters and the 
receiver, and making collision report decisions at the receiver without knowing the communication rates, 
information theoretic analysis has not been fully and successfully extended to practical random access 
systems. Consequently, without the support of rigorous coding theorems, standard networking practice 
often focuses on throughput optimization using packet-based channel models [10]. The explicit connection 
of the packet-based channel models to the physical layer channel is usually not specified except through 
the limited means of packet erasure channels. Networking practice allows bursty traffic arrivals and does 
not require joint communication rate determination among transmitters. However, the use of abstracted 
packet-based communication model and the lack of rigorous coding theoretic analysis essentially prevent 
an insightful understanding about the impact of physical layer communication to upper layer networking 
[8]. 

In this paper, we propose an approach that holds promise in extending information theoretic analysis 
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to packet random access systems with bursty traffic. Tlie essence of our approacli consists of using the 
classical foundation of coding for each packet and explicitly building-in the support of random access 
operations and bursty traffic phenomena in the following sense. In our coding approach, each transmitter 
determines its communication rate by choosing the number of data bits to encode in each packet. It requires 
neither joint communication rate determination among transmitters, nor pre-sharing communication rate 
information with the receiver. It also enables collision detection at the receiver whenever reliable packet 
recovery is not possible. Although defined quite differently from classical channel coding, we find that 
the introduced coding approach does lead to a meaningful achievable rate region characterization that 
is consistent with current understanding and methodology of Information Theory. More specifically, we 
define an achievable region on the communication rates such that reliable packet recovery is supported 
for all communication rates within the region and reliable packet collision detection is supported for all 
communication rates outside the regiorQ. For random multiple access communication over a discrete-time 
memoryless channel using a class of random coding schemes, we show that the achievable rate region 
of the introduced coding approach equals the Shannon information rate region without a convex hull 
operation. Although we only illustrate our results in single-user and simple multiple access systems, the 
general problem formulation shown in the paper can be extended to other random access scenario]^. 

Next, we start with a detailed explanation of the coding approach in a single user system (i.e., single 
transmitter-receiver pair) in Section |Ill We then extend it to a random multiple access system and prove 
the main coding theorem in Section [nil Further extensions are discussed in Section |IVl 

II. A New Packet Coding Approach - The Single User Case 

Let us first consider a single user communication system over a discrete-time memoryless channel. 
The channel is modeled by a conditional distribution function Py\x where X E X is the channel input 
symbol and Y E y is the channel output symbol and, X, y are the finite input and output alphabets. 
We assume that time is slotted with each slot equaling symbol durations, which is also the length 
of a packet. Unless otherwise specified, we will confine our focus on block channel codes of length 
that represent coding within each packet. Throughout this section, we assume communication channel is 
time-invariant. The channel is known at the receiver but unknown at the transmitter. Our main objective 
is to use a relatively simple system model to introduce the basic coding approach that provides multiple 
rate options to the transmitter and enables collision detection at the receiver. Proofs of the claims and 

'Note that communication rates are chosen arbitrarily and the rate information is unknown at the receiver. 

^We want to emphasize that our work does not purpose to fully bridge the gap between Networking and Information Theory. However, 
it does provide a useful link between rigorous communication rate determination and practical random access networking. 
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theorems given in this section are skipped since they are trivially implied by the more general theorems 
given in Section Unl 

A. Random Coding with Multiple Rate Options 

Consider the simple case when the transmitter uses a classical random coding scheme, originally 
introduced in [11]. The coding scheme is described as follows. Let C = {Cg : 6* G 6} be a library 
of codebooks, indexed by a set 0. Each codebook contains 2^^" codewords of length A^, where Rq is 
a predetermined rate parameter. Denote \C0{w)]j as the jth symbol of the codeword corresponding to 
message w in codebook Ce- Assume at the beginning of a time slot, the transmitter randomly generates 
codebook index 6 according to a distribution 7. The distribution 7 and the codebooks Ce are chosen such 
that random variables Xyjj : 6 — )■ [C0{w)]j, yj,w are i.i.d. according to a predetermined input distribution 
Px- We assume the receiver knows the code library and the value of 9, that is, the receiver knows the 
randomly generated codebook. This can be achieved by sharing a random codebook generation algorithm 
with the transmitter. Based upon this information and the channel output, the receiver gives an estimate w 
of the transmitted message w. Define Pe{w) = Pr{w ^ w} as the decoding error probability given that 
w is the transmitted message. By following the analysis in [11], it is easily seen that, if _Ro < H^'j 
there exists a sequence of decoding algorithms to achieve lim7v->oo -Pe(^'^) = for all w. The asymptotic 
result here should be interpreted as: given two small positive constants ei, 62, there exists a threshold 
A^(ei,e2), such that > iV(ei,e2) and Rq < /(X; F) - £2 imply Peiw) < ei for all w [12]. Although 
understanding of the tradeoff between A^, ei, 62 is important, it is outside the scope of this paper. We will 
not repeat this well-known observation in the rest of the paper. 

Now recall the standard practice of packet networking with bursty traffic [9]. Depending on message 
availability, in each time slot, the transmitter will either stay idle or transmit a packet according to data 
availability and the MAC layer protocol. Suppose the same coding scheme is used in multiple time slots, 
which means that, when channel code is designed, the transmitter does not know whether or not a message 
will be available in a particular time slot. To model the idle operation, we regard "idle" as a specific channel 
input symbol and insert a particular codeword Cg{l) = {idle, . . . , idle} into every codebook in the library 
C When no input data is available, we say w = I and the transmitter sends Cg{l) through the channel. 
It can be shown that, if Rq < /(X; Y), we can still achieve hm/vr^^oo -Pe(^i^) = for all w. 

Based on the above coding scheme, we will now introduce a communication rate parameter r. According 
to the usual practice, when the transmitter idles, we say the communication rate is r = 0, otherwise, the 
communication rate is r = Rq. We say the codebook has two classes of codewords. The first class contains 
one codeword Cg{l) corresponding to r = 0. The second class contains 2^^° codewords corresponding to 
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r = Rq. The coding scheme enables the transmitter to choose its communication rate r G {0, Rq} without 
sharing the rate information with the receiver. If Rq < I{X; Y), reliable recovery of the {w,r) pair can 
be achieved asymptotically. 

Next, let us consider a more complicated situation. Assume each codebook in the library contains 
three classes of codewords. As before, the first two codeword classes contain one and 2^^° codewords, 
respectively. The third class contains 2^^^ codewords corresponding to r = We assume the index 
distribution 7 and the codebooks are designed such that random variables X^j : 9 [Ce{w)]j, ^j,w 
are independently distributed. Codeword symbols in the second class are i.i.d. according to an input 
distribution Px\Ro^ while codeword symbols in the third class are i.i.d. according to an input distribution 
Px\Ri - Note that the two input distributions may be different. Let Ir^^{X] Y) and /r^(X; Y) be the mutual 
information between channel input and output symbols computed using input distributions Px\Ro and 
Px\Ri, respectively. Assume Rq < Ijig(X;Y), while Ri > In this case, the coding scheme 

provides three rate options r G {0,-Ro;-Ri} at the transmitter. However, it is clear that reliable message 
recovery is not always possible. Nevertheless, rather than trying to recover the messages blindly, let us 
assume the receiver intend to achieve a different set of objectives. For all messages corresponding to r = 
and r = Rq, the receiver intends to recover the messages. For all messages corresponding to r = the 
receiver intends to report a packet "collision'!^. Note that the receiver needs to achieve these objectives 
without knowing the actual communication rate r. To be more specific, let (w, r) be the transmitted 
message and rate pair. The receiver either outputs an estimated pair (w),f), or outputs a "collision". For 
(w, r) with r G {0, Rq}, we define Pe{w, r) = Pr{(w, r) 7^ {w, r)} as the decoding error probability. For 
{w,r) with r = Ri, we define Pc{w,r) = PrjcoUision} as the collision detection probability. It can be 
shown that, there exists a sequence of decoding algorithms to asymptotically achieve limAr_^oo Peiw, r) = 
for all (w, r) with r G {0, Rq}, and limAr^oo Pciu), r) = 1 for all {w, r) with r = Ri. 

In the above coding scheme, the transmitter has multiple rate options. The encoding scheme can be 
designed with only channel alphabet information. If the receiver knows the communication channel, 
whenever the transmitter sends a message with rate r < Ir{X; Y), the receiver can asymptotically recover 
the message with an error probability arbitrarily close to zero; whenever the transmitter sends a message 
with rate r > Ir{X;Y), the receiver can asymptotically report a collision with a probability arbitrarily 
close to one. 

'Note that the term "colhsion" is used to maintain consistency with the networking terminology. Throughout the paper, "collision" means 
packet erasure, irrespective whether it is caused by multi-packet interference or not. 
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B. Generalized Random Coding Scheme and The Standard Rate 

In the previous section, we defined communication rate as the number of data bits per symbol encoded 
in a packet. The transmitter uses a random coding scheme whose codewords are categorized according to 
different rate options. In each category, codeword symbols are generated to be i.i.d. Asymptotic behavior 
of the system is also categorized according to the communication rates. However, if we view the coding 
scheme from a different angle, we will see that the communication rate parameter r is actually a subtle 
concept. Since the transmitter essentially uses a codebook to map a message and rate pair (w, r) into 
a codeword, one can index all possible values of (w,r), and regard the index as a macro message. 
Consequently, the communication rate r = r{W) becomes an arbitrarily defined function of W , or in 
other words, r is determined by an arbitrary partitioning of the codebook. Properties of the achievable 
rates are dependent on the particular form of the r{W) function. For example, if all codeword symbols in 
the random coding scheme are generated according to the same input distribution, different rate definitions 
(or codewords partitioning) lead to different rate achievability results even though they all correspond to 
the same codeword generation algorithm. To avoid such ambiguity, we introduce a standard rate function 
r{W) as follows. 

Definition 1: (standard communication rate) Assume codebook C has 2^^'"="' codewords of length 
A^, where -Rmax is an arbitrary large finite constant. Let the corresponding messages or codewords be 
indexed by W E {1, . . . , 2^^™='"}. For each message W, we define its standard communication rate, in 
bits per symbol, as r{W) = logg W. ■ 

Unlike the practical communication rate definition used in Section III-A[ the standard rate function 
r(W) = log2 W is invertible. Consequently, achievable rates of any other rate function can be derived 
from achievable rates of the standard rate function. 

Next, we introduce a generalized random coding scheme, in which, symbols of different codewords, as 
opposed to different codeword classes, can be generated according to different input distributions. 

Definition 2: (generalized random coding) Let C = {Ce : 6* G 0} be a library of codebooks. Each 
codebook in the library contains 2^^™^" codewords of length A^, where -Rmax is an arbitrary large finite 
constant. Let the codebooks be indexed by a set 0. Let the actual codebook chosen by the transmitter 
be Ce where the index is a random variable following distribution 7. We define (£, 7) as a generalized 
random coding scheme following distribution Px\w, given that the following two conditions are satisfied. 
First, the random variables Xi^j : 9 — t- [C6i(iy)]j, Vj, W, are independent. Second, given W, X^/j should 
be i.i.d. according to input distribution Px\w- ■ 

We now define a sequence of generalized random coding schemes that follows an asymptotic input 
distribution. 



Definition 3: Let {(C^^\ 'j^^^)} be a sequence of random coding schemes, where {C^^\'y^^^) is a 
generalized random coding scheme with codeword length and input distribution P^^^jj^) • Assume each 
codebook in library C^^^ has 2^^™="" codewords. Let Px\r be an input distribution defined as a function of 
the standard rate r, for all r G [0,-Rmax]- We say {{C^^\'y^^^)} follows an asymptotic input distribution 
Px\r, if for all {11/*^^^} sequence^ with well defined rate limit ^iW^^^), we have 

Note that since we do not assume Px\r is continuous in r, we may not have hmx^oo Px\r{w('^^) = 

-Pxllimiv^<^r(W^W)- ■ 



C. Achievable Rate Region of Single-user Random Access Communication 

Consider single-user random access communication using a generalized random coding scheme (C, 7) 
with codeword length A^. In each time slot, the transmitter randomly generates codebook index 9 and uses 
codebook Ce in the library to map a message W into a codeword. Assume receiver knows the channel 
and the codebook Cg, based on this information and the received channel symbols, the receiver either 
outputs a message estimate W or reports a collision. Let Pe\g{Cg, W) = Pr{W 7^ W} be the probability, 
conditioned on 9, that the receiver is not able to recover message W. Let Pc\e{Cg,W) = PrjcoUision} 
be the conditional probability that the receiver reports a collision. We define Pe{W) and P^W) as the 
unconditional error probability and the unconditional collision probability of message W, respectively, as 
follows 

P,iW) = Ee[Peie{Ce, W)], P,{W) = Ee[Pcie{Ce, W)]. (2) 

With the error probability definition, we define an achievable rate region of the system as follows. 

Definition 4: Consider single user random access communication using a sequence of random coding 
schemes {{C^^\'j^^^)}, where {C^^\'j^^^) is a generalized random coding scheme corresponding to 
codeword length N. Let R C [0, -Rmax] be a region of standard rates. Let Rc be the closure of R. We 
say R is asymptotically achievable if there exists a sequence of decoding algorithms under which the 
following two conditions are satisfied. First, for all message sequences {ly*^^^} with r{W^'^'^) E R for 
all and limAr^oo '"(W^*^^^) G R, where r{W^^^) = -^\og2W^^^ is the standard rate function, we have 
limN^oc Pe{W^^^) = 0. Second, for all message sequences 

with r(iy(^)) Rc for all N and 

lim^^oor(iy(^)) ^ R„ we have Mm^^^ P,{W^'''>) = 1. M 

We are now ready to formally present the coding theorem for a single-user random access system. 

''Here {W^'^'} refers to an arbitrary deterministic, i.e., not random, message sequence. Each message W*^' in the sequence takes value 
in the index set {1, . . . , 2^'f*»='-}. The standard rate function r(W^(^') is defined as r{W^^>) = i logj W^^l 
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Theorem 1: Consider single user random access communication over a discrete time memoryless 
channel Py\x- Assume the transmitter is equipped with a sequence of generalized random coding schemes 
7*^^^)} following an asymptotic input distribution Px\r- Assume Px\r is continuous in r except at 
finite number of points. The following standard communication rate region R is asymptotically achievable, 

= {r|either r = or r < /^(X;r)}, (3) 

where mutual information IriX; Y) is computed using input distribution Px\r- 

Furthermore, for any sequence of random coding schemes following asymptotic input distribution Px\r, 
assume R is an asymptotically achievable rate region. Let f be an arbitrary rate inside R in the sense that 
we can find 6 > with r E R for all f < r < f + 5. If the asymptotic conditional input distribution Px\r 
is continuous in r at f, then we must have 

r<Ir{X;Y). (4) 

■ 

Theorem [T] is implied by Theorem [2] given in Section [nil 

It is important to note that the achievable rate region characterized in Theorem [T] is significantly different 
from a Shannon information rate region [1]. For reliable communication in the classical Shannon sense, 
information rate equals the normalized log of the codebook size. The rate must be predetermined since 
both the transmitter and the receiver need to know the codebook before message transmission is started. 
Collision or erasure detection is therefore not needed if the receiver knows the channel [2]. In our coding 
approach for random access communication, the codebook contains a large number of codewords. With 
a new definition of the communication rate, either in the practical sense as used in Section III-AI or in the 
standard rate sense as introduced in Section III-Bl the coding scheme provides multiple rate options to the 
transmitter. If the transmitter does not know the channel and chooses its communication rate arbitrarily, it 
becomes the receiver's responsibility to detect whether the communication rate is outside the achievable 
rate region or not. This is a fundamental functionality required in random access communication but not 
seen in classical Information Theoretic communication. 

By regarding collision report at the receiver as message erasure, the new coding approach shares certain 
similarity with error and erasure decoding [13] in a classical communication system. However, there is 
also a fundamental difference. In classical communication, if the codebook is optimized with information 
rate below the channel capacity, then both the error and the erasure probabilities diminish asymptotically 
for all codewords in the codebook. In our coding approach, however, the normalized log of the codebook 
size can be way above the channel capacity. In this case, codewords are essentially partitioned into 
two classes according to their associated communication rate values. For codewords whose rates are 
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Px\r 



inside the achievable rate region, the probability of successful decoding goes to one asymptotically. For 
codewords whose rates are outside the region, however, the probability of message erasure goes to one 
asymptotically. Such codeword categorization and codeword-dependent asymptotic decoding behavior are 
not seen in classical communication systems. 

Note that, in Theorem [H the asymptotic input distribution Px\r could be discontinuous in r. For random 
access communication, the inclusion of a possible discontinuous input distribution Px\r is necessary. 
Consider the last example coding scheme given in Section ITl-AI Each codebook in the library is partitioned 
into three codeword classes. The codeword classes contain 1, 2^^°, 2^^^ codewords, respectively. We have 
explained that, practically, these codeword classes correspond to the options of encoding 0, NRq, and NRi 
data bits into a packet. Suppose the communication channel is Gaussian. For the three encoding options, 
a common choice of the transmitter could be setting the input distributions at Gaussian distributions 
corresponding to three different transmission powers, P = 0, P = Pq, and P = Pi. Let us denote the 
three Gaussian input distributions by A/'(0, 0), A/'(0, P^), A/'(0, Pi). If we characterize the asymptotic input 
distribution as a function of the standard communication rate, we get the following Px\r- 

Ar(0,0) r = 

A/'(0,Po) 0<r<Po (5) 
Ar(0, Pi) Ro<r<Ri 
In this example, Px\r is discontinuous in r at r = and r = Rq. 

III. A New Packet Coding Approach - Random Multiple Access Communication 

In this section, we extend the coding theorem to a f^-user, symbol synchronous, random multiple access 
system over a discrete-time memoryless channel. The channel is modeled by a conditional distribution 
function Py\Xi,...,Xk^ where Xi E Xi is the channel input symbol of user i with Xi being the input alphabet, 
and F G 3^ is the channel output symbol with y being the output alphabet. To simplify the discussion, 
we assume y and X^, for all i, are finite. Extending the results to continuous channels is straightforward. 
As in Section HH we assume time is slotted with each slot being equal to symbol durations, which is 
also the length of a packet. 

We assume each user, say user i, is equipped with a generalized random coding scheme (£j,7j). Each 
codebook in the library contains 2^^™^'= codewords of length A^, where Pmax is an arbitrary large constant 
whose particular value is not important. At the beginning of each time slot, user i randomly generates a 
codebook index 6i, and uses the corresponding codebook in its library to encode a macro message Wi into 
a codeword. Note that a macro message contains communication rate information, as already explained 
in Section III-BI We assume the channel is known both at the transmitters and at the receiver. The receiver 
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also knows the particular codebook chosen by each user, and this can be achieved by sharing the random 
codebook generation algorithms with the receiver. However, we assume communication rates of the users 
are shared neither among each other nor with the receiver. 

We use bold-font characters to denote vectors whose ith elements are the corresponding variables of 
user i. For example, C represents the vector of code libraries of the users. Also, 6 denotes the random 
index vector, Cg denotes the codebook vector, W denotes the message vector, r(W) denotes the standard 
rate vector, and Px\r denotes the asymptotic input distributions of the random coding schemes, etc. 

A. Collision Detection for All Users 

Let W be the transmitted message vector, encoded using codebook Ce- Assume, upon observing the 
received channel symbols, the receiver either outputs message estimate W for all users, or reports a 
collision. We define Pe\0{Cg, W) — Pr{W ^ W} as the probability, conditioned on 6, that the receiver 
is not able to recover the message vector W . Define Pc\b{Cq^ W) = Pr{collision} as the conditional 
probability that the receiver reports a collision. Assume random coding schemes (>C,7). Let Q be drawn 
independently according to 7. We define PSW) and PjyV) as the unconditional error probability and 
the unconditional collision probability of message W , respectively. 

P,{W) = Ee{Pe\B{Ce, W\ P,{W) = Ee{P,\e{Ce, W)\. (6) 

Definition 5: Consider random multiple access communication using a sequence of random coding 
schemes {(>C''^\7'^^))}, where {Cy^\-^'^^^) is a vector of generalized random coding schemes with each 
codebook in CP^^ containing 2^^™'^ codewords of length N . Let i? be a region of standard rate vectors. 
Let Re be the closure of R. We say Tl is asymptotically achievable if there exists a sequence of decoding 
algorithms under which the following two conditions are satisfied. First, for all message sequences 
with r{W^^^) G R for all N and limjv^oo viyV^^^) E R, we have Mm^^^ PeiW^^^) = 0. Second, for 
all message sequences {W^^^ with r{W^^^) ^ Rc for all N and limN^^r{W^^^) ^ R^, we have 

lim^_ooPc(W^^'^^) = l. ■ 

The following theorem characterizes the asymptotically achievable rate region of a random multiple 
access system. 

Theorem 2: Consider random multiple access communication over a discrete-time memory less channel 

Py\Xi Xk using a sequence of random coding schemes {{C'^^\^^^^)}. Assume {{C^^\^^^^)} follows 
asymptotic distribution Px\r- Assume that, for any user i, Pxiin is only discontinuous in at a finite 
number of points. The following standard communication rate region R is asymptotically achievable. 



R 



\r WS C{1,...,K}, either n^s = 0, or ^r, < /^(X.es; Y\X,^s) ] , 
I ies ) 



(7) 
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where mutual information /^(Xjgs; Y\Xi^s) is computed using input distribution Px\r- 

Furthermore, for any sequence of random coding schemes following the asymptotic conditional input 
distribution Px\r, assume R is an asymptotically achievable rate region. Let r be an arbitrary rate vector 
inside R in the sense that we can find S > with r E R for all f < r < r + 61, where 1 is a vector of 
all ones. Let S* C {1, . . . , K} be an arbitrary user subset. If the asymptotic conditional input distribution 
Px\r is continuous in r^gg at f, then we must have 



The achievability part of Theorem [2] is implied by Theorem [3] which is provided later in this section. 
The proof of Theorem [3] is given in Appendix |B] and the converse part of Theorem |2] is proved in Appendix 



When the asymptotic conditional input distribution Px\r is not a function of r, i.e. codewords of each 
user are generated according to the same input distribution, the achievable rate region R given in (|7]) 
becomes 



This is identical to the Shannon information rate region without a convex hull operation for the multiple 
access channel under a given input distribution [2]. 

Note that in a classical channel coding scheme, users jointly determine their information rates by 
choosing codebooks with appropriate numbers of codewords. To decode the messages, the receiver 
essentially needs to search all codewords (e.g. maximum likelihood decoding or typical sequence decoding) 
throughout the codebook [2]. Random access communication does not assume joint rate determination 
among users [10]. Therefore the users cannot know a priori if their rate vector is within the achievable rate 
region or not. In this sense, the situation is similar to the single-user case discussed in Section HIl In our 
coding approach, codebooks of the users contain large numbers of codewords. However, the codewords 
are indexed by their standard rate parameters and the receiver only searches for appropriate codewords 
within the achievable rate region, which can be regarded as a subset of joint codewords in the codebooks. 
The receiver reports a collision if an appropriate codeword cannot be found. 

B. Collision Detection for Each Individual User 

In Section IIII-A[ we assumed that the receiver should either output reliable message estimates for all 
users, or report a collision. When multiple users compete for the channel, it is often the case that the 
receiver may only be interested in recovering messages from a subset of users. Correspondingly, a collision 



Y,r,<UiX,es;Y\X,^s)- 



(8) 



El 




(9) 
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should only be reported when messages from the users of interest are not decodable. Let us first assume 
the receiver only wants to recover the message from a particular user, say user k. Given that message W 
is transmitted over the multiple access channel using codebook Cq. The receiver either outputs a message 
estimate Wk for user k, or reports a collision. 

Before we proceed further, it is important to note that, even though the receiver is only interested 
in decoding the message of user k, whether reliable message recovery is possible for user k still can 
depend on the transmitted messages of all users. Therefore, when we define the error probabilities and 
the achievable rate region, the rates of all users are still involved. 

Define PekieiCe, W) = Pr{Wk ^ Wk} as the probability, conditioned on 6 that the receiver is not 
able to recover user A;'s message Wk- Define Pck\e{Ce, W) = PrjcoUision} as the conditional probability 
that the receiver reports a collision for user k. We define Pefe(W^) = Ee[Pek\e{'^e, VF)] and Pck{W) = 
Ee[Pck\e{(^e, W)] as the error probability and the collision probability of message W for user k. 

Definition 6: Consider random multiple access communication using a sequence of random coding 
schemes {{C^^\^^'^^)}, where {C^^\^^^^) is a generalized random coding scheme with each codebook 
in C^'^^ containing 2^^™'''' codewords of length A^. Let R^. be a region of standard rate vectors. Let R^c be 
the closure of Rk- We say Rk is asymptotically achievable for user k if there exists a sequence of decoding 
algorithms under which the following two conditions are satisfied. First, for all message sequences 
with r(W(^)) G Rk for all N and MmN^oo r{W^^^) G Rk, we have MmN^^ P^kiW^^^) = 0. Second, 
for all message sequences {V^^^''} with r(Vr^^-') ^ Rkc for all N and limN^oo r{W^^'^) ^ Rkc, we 
have \imj,^^P,k{W^''^) = l.m 

The following theorem gives an achievable rate region for user k. 

Theorem 3: Consider random multiple access communication over a discrete-time memoryless mul- 
tiple access channel Py\x^,...,Xk using a sequence of random coding schemes {{C^^\^^^^)}. Assume 
{(Z^*-^-*, 7'-^^)} follows asymptotic input distributions Px\r- Assume that, for any user i, Pxiin is only 
discontinuous in r-i at a finite number of points. The following standard rate region R^ is asymptotically 
achievable for user k. 



where mutual information Ir{X-^g; Y\Xi^s) is computed using input distribution Px\r- ■ 
The proof of Theorem [3] is given in Appendix El 

Theorem [3] can be easily extended to the case when the receiver is interested in recovering messages 
from a subset of users, as shown in the next theorem. 

Theorem 4: Consider random multiple access communication over a discrete-time memoryless mul- 
tiple access channel Py\Xi,...,Xk using a sequence of random coding schemes {{C^^\j^^^)}. Assume 




or 3S C S,k e S, such that, J^ies < ^riX^^g; Y\Xi^s) 



(10) 



R 



'So 



(11) 
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{{C^^\^''^^)} follows asymptotic input distributions Px\r- Assume that, for any user i, Pxiin is only 
discontinuous in rj at a finite number of points. Let Sq C {1, . . . , K} be a user subset. The following 
rate region Rs^ is asymptotically achievable for users k E Sq. 

^sc{i,...,K},snSo7^(p,3S,snSoCSc s, 

such that, either r-^g = 0, or Eigs^^i < UiX^^g-, Y\Xi^s) 
where mutual information Ir{Xj^g] Y\Xk(^s) is computed using input distribution Px\r- ■ 
The proof of Theorem |4] is given in Appendix |Dl 

Note that when Sq = {1, . . . , K}, i?5,) given in (fTTI) is equal to the rate region R given in 

IV. Simple Examples 

In this section, we illustrate the asymptotically achievable rate region results in two simple example 
random multiple access systems. 

Example 1: Consider a K user random multiple access system over a memory less Gaussian channel 
modeled by 

K 

Y = J2X^ + V. (12) 

i=l 

where is a white Gaussian noise with zero mean and variance Nq. 

Assume that the input distribution of user k, for all k, is Gaussian with zero mean and variance P^, 
irrespective of the rate parameter. Assume the receiver wants to recover messages of all users. According 
to Theorem [T} the following rate region is asymptotically achievable 



R 



V5C{l,...,ir},;^r,<ilogfl + %^) \. (13) 



Note that the achievable rate region R is identical to the Shannon channel capacity region [2]. 

If for all k, the input distribution is Gaussian with zero mean and variance Pk for any non-zero rate, 
and user k idles at rate zero, then the achievable rate region is still given by (fT3l) . 

Example 2: Consider a K user random multiple access system over a memoryless symbol collision 
channel. We define an nth order symbol collision channel as follows. The channel input alphabet of any 
user is given by A" = {0, 1, ... , 2"}, where represents an idle symbol. The channel output alphabet is 
given by 3^ = {0, 1, ... , 2", c}, where c represents a collision symbol. If all users idle, the receiver receives 
an idle symbol, F = 0; if only one user, say user k, transmits a non-zero symbol Xk, the receiver receives 
Y = Xk, if multiple users transmit non-zero symbols, the receiver receives Y = c, i.e., a collision symbol. 
We assume in all input distributions the non-zero symbols always take equal probabilities. Consequently, 
an input distribution Px^lvk '^^^ characterized through a single parameter p{rk), which is the probability 
that any particular symbol in the transmitted codeword takes a non-zero value. 
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Proposition 1: Assume the conditional input distribution of user k, for all k, is given by 




1 - \Jrk/n for = 



forX,, =je {1,...,2"} 



(14) 



In other words, p^r^) = yrk/n. Assume that the receiver wants to recover the messages of all users. The 
following rate region R is asymptotically achievable. 



Proposition [T] is proven in Appendix El 

Note that when K = 2, the rate region R given in (fTSl) equals the random multiple access throughput 
and stability regions of the collision channel [14], which also approaches the asynchronous information 
capacity region of the collision channel as n — )► oo [15]. Proposition [H therefore motivates the question 
whether there is a fundamental connection between the achievable rate region studied in this paper and 
the throughput, stability, information capacity regions of the random multiple access channel discussed in 
[10]. Obtaining a theoretical answer to this question is an important issue for future research. 

In both examples, the achievable rate region R is coordinate convex in the sense that if f G then 
r G -R for all r < r. However, in general, depending on the choice of the asymptotic input distribution, 
the region may not be continuous in the sense that not every point pair in the region is connected by a 
continuous path in the region. In this case, the region is certainly not coordinate convex. 



Throughout our analysis, we obtained asymptotic results by taking the packet length to infinity. It 
is unfortunately the nature of random access systems that their packets are often reasonably short. 
Therefore, it is necessary to investigate the achievable rate and error performance under the assumption 
of a finite packet length. This is similar to the channel dispersion analysis presented in [12] for classical 
communication systems. Alternatively, one can also characterize the achievable rate and error performance 
tradeoff in a way similarly to the error exponent analysis for classical channel coding [3]. An example of 
such derivation can be found in [16]. 

We have assumed that the multiple access channel is fully known at the receiver. Even though users do 
not pre-share their rate information, their codebooks are assumed known at the receiver. To extend our 
results to other random access scenarios, one has to carefully examine whether these assumptions are still 
valid or reasonable. If not, deriving coding performance under various channel and codebook information 
availability assumptions becomes the key challenge. 




(15) 



V. Further Discussions 
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The coding approach studied in this paper enabled a network user to choose its communication rate by 
encoding a variable number of data bits into a packet. How should a user determine its communication 
rate in a particular time slot, however, is a challenging MAC layer problem. Nevertheless, comparing 
to the packet-based channel model used in classical MAC layer performance analysis [10], performance 
limitations of the introduced coding approach provided a useful bridge to support the joint optimization 
of MAC layer networking and physical layer channel coding. 

VI. Conclusion 

We proposed a new channel coding approach for coding within each packet in a random access system 
with bursty traffic. The coding approach enabled each user to choose its communication rate without 
pre-sharing the rate information with the receiver and other users. Performance of the coding approach 
is characterized by an achievable region defined on the communication rates. The receiver is required 
to output reliable message estimates if the communication rates are inside the achievable rate region; 
the receiver should report a collision if the rates are outside the region. We showed that the maximum 
achievable rate region of random coding schemes takes a form similar to the Shannon information rate 
region without a convex hull operation. The achievable rate region when the receiver is interested in 
recovering messages only from a subset of users is also obtained. 

There are numerous questions left open that would further tighten the connection between random access 
networking and Information Theory. We believe that our approach contributes an important component to 
that connection by distinguishing the issues of reliable communication and reliable collision detection in 
a rigorous manner. 

Appendix 

A. Proof of Theorem |2] 

Proof: Note that the achievability part of Theorem [2] is implied by Theorem [3l Here we prove the 
converse part of Theorem |2l 

Let R be an asymptotically achievable rate region. Let f G -R be a rate vector inside R. We can find 
a 5 > such that r E R for all r < r < f + 51. Let 5 C {1, . . . , K} be a given user subset. 

Because the asymptotic distribution Px\r, and hence the entropy functions, are continuous in rjg^ at 
f, we can find a small enough 6 > and a bound > with lim^^o us = 0, such that for all rates r, 
with Vi^s = f-i^s and Vi^s < ^ies < ries + ^hes, we have 

\Ir{Xies; Y\Xi^s) - If{Xies; Y\Xi!^s)\ < us, (16) 
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where mutual information /^(Xjg^; y is evaluated using input distribution Px\r and mutual 

information Ir{Xi^s] Y\Xi^s) is evaluated using input distribution Px\r- 

Let W^^^ be the actual source message. We assume W^'^^ is generated randomly according to a uniform 
distribution under the condition that Vi^si^^^^) = ^^'^ "^ies < ri(^s{W^^^) < f-jg^ + ^lje^. Assume 
codewords X^^g are known to the receiver. Let "* be the message estimated by the receiver. Define 
piN) ^ Pr{W^^^ ^ VF^^)} as the error probability. Note that 

lim = ^ E log2 (2^^^+^ - r^) = T.i^^ + S). (17) 

We assume N is large enough such that Hiyv'f^g) > NJ^ies^i- According to Fano's inequality [17], we 
have 



i&S 



< 1 + ^pr)i/(w^S) + ^/(^SS; (18) 



N N ^ V ^ 
For large enough A^, (fTSi) implies 



i:f.<^/(W'g;W'.'SKi) + «. 

iG5 



</^.(X,65;l^|^i^5) + 2M5. (19) 



Taking 5 ^- in yields 



B. Proof of Theorem |J] 

Proof: Although the basic idea of the proof follows from Shannon's typical sequence arguments 
introduced in the channel capacity proof [1], it is significantly complicated by the following two factors. 
First, the receiver is only interested in recovering a particular user's message rather than messages of 
all users. Second, the empirical input distribution of different codewords may be different. Due to these 
complications, and for easy understanding, we choose to present the full proof in five detailed steps. 

Step 1 (The typical sequence decoder): We first define a typical sequence decoder as follows. 
Consider a sequence of random coding schemes {(Z^'-^-*, 7^^^)} following asymptotic input distribution 
Px\r- For any given codeword length N, let X*^^^ be the randomly chosen codeword associated to 
message W^^\ Let Y^^^ be the channel output sequence. Define r{X^^^) = r{W^^'^) = log2(W^^^) 
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as the standard rate vector corresponding to W^^^ and X^^\ To simply the proof, we assume the input 
distribution of random coding scheme {C^^\'y^^^), denoted by f x indeed equal to the asymptotic 

input distribution -Px|r(w('^))' Extending the proof to the general case is straightforward. 

We define the set Al,^\P (^x ,Y)\r) of jointly typical sequences with respect to the 

distribution P(^x,Y)\r — PY\xPx\r as the set of length N sequences with empirical entropies e-close to 
the true entropies. 

ysc{i,...,K}, 



-j^log,Pr{X%l}-HiX,^s) 

log2 Pr{X%l - H{X,,s, Y) | < e} , 



(20) 



where the entropies are computed using input distribution P(^x,Y)\r- 

Assume after observing the channel output sequence Y^^\ the receiver obtains a set X'^^'^ of codewords 
that are jointly typical with Y'^^\ 

) = {x(^) I (xw, y W) e Af){P^x,Y)\rl r-(xW) eR,]. (21) 

The receiver outputs message estimate W^^^ for user k if, for all codewords X*^^^ G X'^^\ the A;th 
codeword x'^^ of X*-^^ G X^^^^ corresponds to the same message wl^\ If such W^^^ doesn't exist, the 
receiver outputs a colUsion for user k. 

Step 2 (Some key definitions): Let cr > be a small constant. We define as a subset of R^. 

<z {I, . . . ^K},k e S,3S ^ S,k e S, 
such that, Eies'^i < ^f-i^ies^ y\Xi^s) - 
Let M be a large integer whose value will be specified soon. We define |o, ^ff^, ■ ■ ■ , ^max} as 



(22) 



the set of grid rates of a user. Define rg{W^^^) 



^max 



M 



as a function that outputs the largest 



grid rate less than or equal to r(W^^)). Let rg(W^^^) be the corresponding vector function of rg{W^^^). 

Let {ly*^^^} be an arbitrary message sequence satisfying lim]\f^^ r(\V^'^'^) = r, for some r. Assume 
W^'^^ is the transmitted message with codeword length N. Let the actual codeword associated to W^^^ 
be X(^), which is a random variable depending on the codebook index 6. Let the channel output sequence 
be Y^^\ It is easy to show 



(23) 
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That is, with an asymptotic probability of one, the receiver will find F(^) be jointly typical with the 
transmitted codeword vector X^^\ 

Step 3 (Error type I, outputting the wrong message): In this step, we will show the probability 
that the receiver can find another codeword vector W in the achievable rate region, i.e., r{W ) G Rk, 
with wj:^^ 7^ Wk^\ is asymptotically zero. 

Without loss of generality, we focus our discussion on an arbitrary subset of messages in the achievable 
rate region. The subset is denoted by B^g^! , illustrated in Figure [TJ with the notation being explained as 
follows. Superscript denotes the codeword length. Subscript S* is a subset of users S C {1, . . . , K} 

wig, and 



with k E S. We assume for all messages VF*"^"* G b'^^J in the subset, we have W^^g 



(N) 



7^ wl^\ Vi G S. In other words, for all users not in S, their messages should equal to the 



corresponding transmitted messages, and for any user in S, its message should not equal to the transmitted 
message. Subscript f is a vector characterizing the rates of messages in the subset. For all messages 
W^'^^ G Bg^^, we have r{w\^l) = r-i^s, and f'g{w'^^l) = f jgs. In other words, entries of Vi^s take 
grid rate values, with fjg^ < r{wl^l) < '^■ies + ^^fi^'^ies- In summary, the message subset Bg^^ is 
defined as follows. 



B 



(N) 
S,r 



w 



w 



(TV) 



(TV) 



(24) 



Let M be large enough, such that the following inequality is satisfied for all messages W^^^ G B^^^ 



and for all user subset S C S, k e S. 

I 



Y\X 



- h{Xi^s] Y\X 



a 



where a is the small constant appeared in (l22ll^l. We also let M be large enough such that 



M 



< 



a 



(25) 



(26) 



Define P^^^B^^^} as the probability that the receiver can find a message W^''' ' G -85^^ whose code 



word X^""^ satisfies {X^''\ F^^)) G ) 



"^(X,y)|r(X*^') 



We will show next that lim^^oo P^^\Bf)} 



0, for all f, S, and r that satisfy the assumptions. 

Note that limjv^oo ^i^H^i^^} = holds trivially in any of the following two situations. First, B^^^ 
is an empty set for large A^. Second, for large and for all W^^^ G -B^f'*, 

^Note that, if the asymptotic input distribution Px\r continuous in r^gs at r for all r satisfying Vn^s = 'i'n^s, ^ies < ^ies < 

figs + ^'m" lies, then i22i may not be satisfied only by setting M at a large value. In this case, one needs to further partition the set 
Bg^' into a finite number of subsets, such that ( 122b is satisfied by messages within each subset. The partitioning is possible because the 
asymptotic input distribution of every user is discontinuous at most at a finite number of rate points. The rest of the proof can essentially 
be applied to each subset of messages with only minor revisions. Hence the corresponding discussion is skipped. 
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Fig. 1. An illustration of achievable rate region and grid rates. 



— j^ log2 Pr{Y'^^^\Xi^g} — H^^^(N)^{Y\Xi^s) > where F^^^ is the channel output corresponding to 
the actual message W^^\ H^^^(N)^{Y\Xi^s) is the entropy function computed using input distribution 



^{x y)|r(w^'^')' Therefore, in the rest of the proof in Step 3, we assume is large enough, B^J^J is not 



empty, and the following inequality holds. 



< e. 



(27) 



Since r G rI^\ we can find a user subset S* C S* with k E S, such that 



J2ri< U{X.^s; Y\Xi^s) - (T- 



(28) 



Let W^^^ E -B^f'' be a message whose codeword is denoted by X^^\ Define r = r('w''^^). We say 



is jointly typical with F^^), denoted by G A'^f^ i ^ (x Y)\r{x''^'') ^^^^^^ 



a codeword X^^^ with {X^^\y^^^) g A(^) (^(x y)|r{x(^))) = MTs- According to the 



definition of Bg\\ for all i E S, W^^^^ ^ Wl;^^\ Under this condition, x\^l and F(^) are generated 
independently. Consequently, the probability that G A[^^ ("^(x Y)|r(x'^')) '^^^ upper 
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bounded as follows. 



(H.iX^^^, Y\X,^s) + e) - {H,{X^^s\X,^s) - e) - {H,{Y\X,^s) - e) 



-N 



IfiX-^s: Y\Xi 



Combining (HH) with we get 



log2 

< -N 
= -N 



3e 



X^-^r<\ Y\X^^ 



a 



a 



3e 



^r{X-^g; Y\Xi^s) ~ 3 ~ 2^ 



Consequently, 



■3e 



Assume is large enough, so that 



Therefore and (gS) yield 

Because Ejgs'^i ~ -^r-(^ies; ^l-^i^s) + o" < 0, choose e < |, we get 

limPr){4^,)} = 0. 



Y^Ti- UiX^^s; Y\Xi^s) + I + 3e 

ies 



(29) 



(30) 



(31) 



(32) 



(33) 



(34) 



Note that, given a, (|34l) holds for all f, 5, and f satisfying our assumptions. Therefore, by taking 
M — i- oo, we can see the probability that the receiver finds a message W^^^ in R^^'' with codeword X^^^ 
being jointly typical with F(^) and wj:^^ ^ ^k'^^ is asymptotically zero. Next, by taking cr — )• 0, we 



conclude that, for all message sequences {VF'-^-'}, the probability for the receiver to output ^ 
is asymptotically zero. As a special case, for all message sequences {VF*^^-*} with r{W^^^) G Rk, VA^, 
and \mvM-,oor{W^''^) G Rk, we have \mvN^^P,k{W^''^) = 0. 

Step 4 (Error type II, failing to report a collision): In this step, we will show if the standard 
rate vector r{X^^^) is outside the achievable rate region, i.e., r{X^^^) ^ Rkc, then the probability 
that the receiver does not report a collision is asymptotically zero. Based on the result we have already 
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demonstrated in Step 3, we only need to show that, asymptotically, the receiver is not able to find another 
codeword vector X inside the rate region, r {X^ )eRk, with Xi^^ = X'f\ and ^ being jointly 
typical with Y^'^\ 

We again focus our discussion on a message set denoted by B^^^ , as illustrated in Figure [H with the 
notation being explained below. As before, superscript N denotes the codeword length. Subscript S* is a 



user subset S C {1, . . . , K} with k E S. Also as in Step 3, we assume for all messages W G B^^ 



AN) 



in the set, Wi^s = ^1^1 and for alH G 5 \ {k}, W}'"> ^ W,^'"'. However, differs from Step 3, we 
= W^.^^ t- Subscript f is a vector characterizing the rates of messages in the subset. For 



(iV) 



assume W, 



all messages W^^'' G Bg^^, we have r{W. 



:<^s\{k}) 



f-i^sMk}, and rgiwl^g^^^^) = ri^s\{k}- 



In other 



words, entries of ries\{k} take grid rate values, with r,(.s\{k} < '^(W^iG5\{fc}) < ^^es\{k} + %^lig5\{fc}- 



In summary, the message set B^J^^ is defined as follows 



B 



(N) 
S,r 



(N) _ 



W 



(N) 



W, 



w, 



(N) 



(TV) 

c 

r{N) 



ytwr',v,es\ik} 

r 



> . 



(35) 



Define -Pi^H-^sf^} as the probability that the receiver can find a message W'"' G B^sV whose 



AN) 



codeword X^^^ satisfies G A(^) 

S, and f, limAT^ooPi'^H^f^^} = 0. 



'^(x,y)|r(x'^') 



. We will show next that, for all r ^ Rkc, 



We let M be large enough, such that the following inequality is satisfied for all messages w'^^^ G b'^^) 



and for all user subsets S C S, k e S 



Y\^i<^S\{k}) — Ir{.X 



ieS\{fc} 



Y\Xi(^s\{k}) 



< 



We also let M be large enough, so that (|26l ) holds. 



(36) 



Note that limjv->oo ^i^H^s f } = holds trivially in any of the following two situations. First, Bg^' is 
an empty set for large A^. Second, for large A^ and for every W^^^ G B^^^! , there exists a user subset Sq 



with SndS 



such that 



-^log2Pr{FW|xSSo} - H^^^im^iY\X,,So, 



of the proof, we assume A^ is large enough, Bg^^ is not empty. For all user subset Sq with So fl 5 



> e. Therefore, in the rest 



Note that in Step 3, we already showed the probability for the receiver to output an erroneous message estimate is asymptotically zero. 
Therefore, in this Step, we only consider the possible situation that the receiver outputs the correct message. 

^As in Step 3, if the asymptotic input distribution Px\r continuous in rigs\{fc} at r for all r satisfying ri^s\{k} = 'i'i^s\{k}, 

^ies\{k} < 'f'ies\{fc} < ■f'iss\{fc} + ■^^f'p^liss\{fe}> then l |36l > may not be satisfied only by setting M at a large value. In this case, one 
needs to further partition the set B^^^ into a finite number of subsets, such that l l36l l is satisfied by messages within each subset. The rest 
of the proof can be applied to each subset of messages with only minor revisions. Hence the corresponding discussion is skipped. 
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we have 



which implies that 



N 



\H,{Y\X,^So)-My\X^^So)\<2^■ 



(37) 



(38) 



Next, we present a key proposition to support the rest of the proof. 

Proposition 2: There exists a user subset Si with Sins' = (p, such that for all user subsets S2 C S U Si, 



with S U Si being the complement of S U Si, we have 



ieS2 



(39) 



The proof of Proposition [21 is given in Appendix | 



Let Si be the user subset found in Proposition |2l Let W^^^ E Bg^ be a message whose codeword 
is denoted by X^^\ Define r = rCW^^^). Combining (|39| ), (|36l) . and (l38l) . we know that, for all user 



subsets S2 C S U Si, the following inequality holds 



J2n + fk> If{Xi^s2U{k}; Y\X^es^) - 2e > IfiX,^s2U{k}; Y\XiesJ - 2e - ^. (40) 

, we know that there exists a user subset S C S, k e S, and a 



Choose e < f . Since r e Ri"\ from ([22 



user subset S2 C S U Si, such that 



iGS2US 

Therefore, from (|40l) and (|4TI) . we obtain 

^ fi < Ir{X i^s~^^f^y, Y\Xi^s\{k}) - Y + 2e. 
je5\{fc} 



(41) 



(42) 



According to the definition of B^^J, for alH G S \ {k}, we have Wi^'^' 7^ W}'"'. Under this condition. 



(iV) 



ie5\{fc} ^^'^ Y^^^ are generated independently. Consequently, the probability that {X 
Ai^^ { P^^ y)|r(x'^') ) upper bounded as follows 



i&S\{k} 



< iV 



(X,y)|r 

-^^■(^te5\{fc}) 



^i^5\{A:}) - e) - {Hr(Y\Xi^s\{k}) - e) 



'^|^i^5\{fe}) — 3e 



(43) 



*Here we regard Si, which is the compliment of Si, as the user subset S in J22t . 
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Combining (|43]) with ([361), we get 



cr 



Consequently, 



w 



(N) 

ieS\{k} 



0,T" 



Assume is large enough, so that 



ies\{k} 



(N) 

iGS\{k} 



w 



Therefore (gi) and (ES) yield 

iog2 (prH4"^}) < iv 



cr 



^ r, - If + - + 3e 

je5\{fc} 



Because Eies\{k} - (^ies\{fc}; y\X^^s\{k}) + (J < 0, choose e < |, we get 



(44) 



(45) 



(46) 



(47) 



(48) 



Note that, given a, (1481) holds for all f, 5, and r satisfying the assumptions. Therefore, by taking 
M — !• oo, we can see, if f ^ Rkc, then the probability that the receiver finds a message W^^^ in R^^^ 
with codeword X^^^ being jointly typical with Y^'^^ and W^.^'^ = W^.^^ is asymptotically zero. Next, 
by taking a — )■ 0, we conclude that, for all message sequences {VF*^^''} with r{W^^^) ^ Rkc, VA^, and 
lim^^oor(iy(^)) ^ Pfc„ we have Mirn^^ PUW^''^) = 1. 

We have now proved that R^ is asymptotically achievable for user k. ■ 



C. Proof of Proposition |2] 

Proof: If the claim of the proposition is not true, then for all user subsets 5*1 with SiHS 



exists a user subset S2 S U Si that satisfies 



ieS2 



there 



(49) 
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Consequently, given the mutually exclusive user subsets S, Si and ^2 with k E S, for all user subsets 5*3 
with k ^ and S'3 fi 5*1 = 6*3 fi S'2 = (j), we have 

= Hf.{X i^s2U{k}) — Hf.{X i^s2U{k}\^ ieSi,'^) 

< Hf.{Xij=S2U{k}) — -f^f(-^iG52U{fc}|^iG5iUS3) ^) 

= Ir{Xi^S2U{k}',y\Xi^SiUS3)- (50) 

From the derivation, we can see that Si U S3 can be constructed to be any user subset satisfying 
•S*! U 5*3 C {1, . . . , K} \ {k}. Consequently, (|50l) implies r E Rkc, where Rkc is the closure of Rk defined 
in (flOl) . This contradicts the assumption that r R^c- Therefore, conclusion of the proposition must hold. 



D. Proof of Theorem |?] 

Proof: Assume we are given a set of rate regions {Rk,k E Sq}, where Rk is asymptotically 
achievable for user k. We will show next that if R^ are defined by (flOl ) for k E Sq, then iigg defined by 
(fTTI) is given by Rs^ = HkeSo Since it is easy to see that Rs^ C flfceSo we only need to show 
Rso ^ rifceso Rk, i.e., r E Rk for all k E Sq implies r E Rso- 

Assume r E Rk for all k E So. Given a user subset 5 fi So 7^ 0, we can find a user subset Si C S and 
SiDSq^ (j), such that 

J2r,<Ir{X^^s,-^Y\X,^s)- (51) 

For the same reason, if S \ Si H Sq ^ (p, we can find a user subset S2 ^ S \ Si, such that 

$:r.</.(X,,^^;F|X^^,^^J. (52) 

In general, if S* \ 5i \ . . . \ (1 Sq ^ cf), we can find 5j C 5* \ 5i \ . . . \ Sj^i, such that 

53 < /r-(^,e5,; ^I^i05\5i\...\5,-i)- (53) 

This procedure can be carried out recursively until for some integer j > 0, S \ Si \ . . . \ Sj D Sq = (p. 
Consequently, define S = {Jk=i ^k- Due to (l53l) . we have 

Er. <i:ir{X,^s,-,Y\X,^s\sA...\s.J = Ir{X,^s;y\X^^s)- (54) 

i£S k=l 

Because S ^ S D So can be found for any 5 C {1, . . . , K}, 5 fl So 7^ 0, we conclude that r E Rso- ■ 
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E. Proof of Proposition \J} 

Proof: Let i?max be the maximum achievable rate region given by Theorem [TJ It is easy to show 
that any rate region R with R C i^max is also asymptotically achievable. Therefore, we only need to 



show that, for all rate vector r with J2iLi yrk/n < 1, the following inequality holds for all user subset 
SC{1,...,K}. 

Y,r,<Ir{X,^s;Y\X,^s). (55) 

ies 

Note that since the channel output symbol F is a deterministic function of the channel input symbol 
vector X, we have Hr(Y\X) = 0. Consequently, 

Ir{Xi(zs:,Y\Xi^s) = Hr{Y\Xi^s) 
> Hr{Y\X,^s = 0)Pr{X,^s = 0} 



Since 



Hr{Y\X,^s = 0) - \/r^/n). (56) 



Hr{Y\X,^s = 0) > - 5] Pr{Y = i} \og^ Pr{Y = i} 

i=l 



> nPr{Y ^ {0,c}} 

>nJ2Pr{X,y^0,Xkesm = O} 

ies 



nY.^Jr^/^ n (l-V^fc/^)' (57) 

ies fc0S\{i} 



we indeed have 



Ir{Xi(.s;Y\Xi^s) > n-^^Ti/nYlil - ^Vk/n) 

ieS k^i 



>nJ2\/ri/n{l-J2yrk/n)>Y.ri, (58) 

ieS k^i ieS 



where the last inequality follows from I]i=i y^k/n < 1. ■ 
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